Method for reducing mosquito noise

ABSTRACT

A method of reducing mosquito noise in a digital image. As mosquito noise is often most plainly visible in the “background” of an image (e.g., the sky or some other backdrop to objects within an image), the luminance value of the background of the image is first determined. Then, the luminances of the pixels of the image are compared against this “background luminance” to determine which should be considered as part of this background. The luminances of these background pixels are then averaged so as to smooth out the representation of the background, and reduce mosquito noise.

BRIEF DESCRIPTION OF THE INVENTION

The present invention relates generally to digital image compression.More specifically, the present invention relates to a method forreducing mosquito noise in MPEG-compressed images.

BACKGROUND OF THE INVENTION

Image compression is often performed in order to reduce the size ofdigital image files. However, the various compression techniques oftenhave associated drawbacks. For example, images compressed according tovarious MPEG formats are known to suffer from “mosquito noise,” or theblurring of the outlines of objects within these images. Named for itsresemblance to the look of mosquitoes flying about the objects of theimages, mosquito noise detracts from the visual effect of anMPEG-compressed picture. Accordingly, continuing efforts exist to reducethe appearance and effect of mosquito noise in compressed images.

SUMMARY OF THE INVENTION

The invention can be implemented in numerous ways, including as a methodand a computer readable medium. Several embodiments of the invention arediscussed below.

As a method of reducing mosquito noise in an image having pixels and arepresentation of a background, one embodiment of the inventioncomprises determining a luminance value corresponding to the background.Luminance values of the pixels are compared to the luminance valuecorresponding to the background, so as to determine a set of backgroundpixels. The pixels of the set of background pixels are assigned averageluminance values determined according to luminance values of the pixelsadjacent to said each pixel.

As a computer readable medium having computer-executable instructionsfor performing a method for reducing mosquito noise in an image havingpixels and a representation of a background, another embodiment of theinvention comprises a first set of instructions for determining aluminance value corresponding to the background. A second set ofinstructions compares luminance values of the pixels to the luminancevalue corresponding to the background, so as to determine a set ofbackground pixels. A third set of instructions assigns, to pixels of theset of background pixels, average luminance values determined accordingto luminance values of the pixels adjacent to said each pixel.

Other aspects and advantages of the invention will become apparent fromthe following detailed description taken in conjunction with theaccompanying drawings which illustrate, by way of example, theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention, together with further objects and advantages thereof, maybest be understood by reference to the following description taken inconjunction with the accompanying drawings in which:

FIG. 1 illustrates process steps for reducing mosquito noise accordingto an embodiment of the invention.

FIG. 2 conceptually illustrates an image pixel and its adjacent pixels.

FIG. 3 illustrates process steps for determining a background luminanceaccording to an embodiment of the invention.

FIG. 4 illustrates process steps for reducing mosquito noise byaveraging luminances of background pixels according to an identifiedbackground luminance.

Like reference numerals refer to corresponding parts throughout thedrawings. Also, it is understood that the depictions in the figures arediagrammatic and not necessarily to scale.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

In one embodiment of the invention, a method of reducing mosquito noisein a digital image is described. As mosquito noise is often most plainlyvisible in the “background” of an image (e.g., the sky or some otherbackdrop to objects within an image), the luminance value of thebackground of the image is first determined. Then, the luminances of thepixels of the image are compared against this “background luminance” todetermine which should be considered as part of this background. Theluminances of these background pixels are then averaged so as to smoothout the representation of the background, and reduce mosquito noise.

FIG. 1 illustrates high-level process steps for reducing mosquito noiseaccording to an embodiment of the invention. While the followingdiscussion relates to a method, it should be appreciated that the blocksor steps illustrated in FIG. 1 (as well as the other Figures) mayrepresent circuitry, modules or devices used to perform the method. Forexample, conventional hardware, software and/or firmware may be used toperform the steps and/or processes discussed herein. It should furtherbe appreciated that such logic steps or processes can be implemented ascomputer-executable instructions stored on a computer readable medium.

First, a background luminance is determined, corresponding to aluminance value for the background as a whole (step 100). Then, thepixels making up this background are identified and their luminances areaveraged (step 102). This averaging yields increased uniformity amongbackground pixels, smoothing out differences between these pixels andtherefore reducing the effect of mosquito noise.

In one embodiment, the determination of a background luminance, and theaveraging of background pixel luminances, are accomplished withreference to a selected pixel and its eight immediately adjacent pixels,as shown in FIG. 2. Here, a grid 200 is shown that illustrates aparticular pixel (i,j) and its surrounding pixels. This pixelconfiguration and its associated nomenclature are used below inexplaining methods of the invention.

FIG. 3 illustrates step 10, the identification of a backgroundluminance, in more detail. In one embodiment, a histogram of all knownluminance values (0-255) is kept. According to methods of the invention,the luminances of certain pixels are added to the histogram, while otherpixels are ignored. When all or a representative number of pixels areanalyzed, the numerically largest histogram value (corresponding to themost frequently encountered luminance value of the non-ignored pixels)is deemed as the background luminance.

Accordingly, the process of FIG. 3 begins (step 300) by setting eachhistogram value to zero (step 302). In embodiments utilizing an array torepresent this histogram, each element of the array can be set to zero.A pixel count can also be kept, and if so this value is set to 1. Afirst pixel (i,j) of the image is then chosen, and the luminances ofthat pixel and each adjacent pixel are determined according to (step304). Typically, luminance values Y(i,j) for each pixel are alreadyknown and simply need to be retrieved, as they are one component of thedata making up the image to be analyzed.

Next, the “energy” (i,j), or sum of the differences between theluminance of the chosen pixel (i,j) and the luminances of each adjacentpixel, is determined (step 306):

$\begin{matrix}{{{Energy}\mspace{11mu}\left( {i,j} \right)} = \frac{\sum\limits_{m = {i - 1}}^{i + 1}{\sum\limits_{n = {j - 1}}^{j + 1}{{{Y\left( {m,n} \right)} - {Y\left( {i,j} \right)}}}}}{8}} & (2)\end{matrix}$where:i=0, 1, . . . , M−1; j=0, 1, . . . , N−1;M=height of the picture in pixels; andN=width of the picture in pixels.

The energy of each pixel (i,j) is then compared to an energy thresholdvalue (step 308). If the energy of this pixel is less than the energythreshold value, the appropriate luminance value of the histogram isincremented (step 310). That is, if pixel (i,j) has a luminance value of150, and its corresponding energy is less than the energy thresholdvalue, the histogram value corresponding to 150 is incremented by 1. Onthe other hand, if the energy of this pixel is greater than the energythreshold value, the pixel is ignored and no value of the histogram isincremented (step 312). In some embodiments, an energy threshold valueof 15 has been found to be effective, although one of skill will realizethat the invention encompasses the use of any energy threshold valuethat assists in identifying background luminances. The process thenmoves on to the next pixel, i.e., increments the pixel count by 1 (step314). If pixels remain (step 316), the process returns to step 306,whereas if no pixels remain, the histogram is examined to determine thelargest of its values, which corresponds to the luminance value sharedby the greatest number of pixels of the image (step 318). This luminancevalue is identified as the background luminance, at which point theprocess terminates (step 320).

One of ordinary skill in the art will observe that this “energy”quantity is an approximate measure of how closely the luminance of aparticular pixel matches with the luminances of its surrounding pixels.That is, the energy of a pixel will be relatively low if surroundingpixels have similar luminance values, and high if surrounding pixelsdiffer greatly in luminance. Pixels with relatively low energy will thusbe similar in luminance to their surroundings. As the background of animage is typically more uniform in color and intensity than theremainder of the image, these low-energy pixels are deemed more likelyto be part of the background. Accordingly, a histogram of low-energypixels is an effective way to keep track of those pixels that make upthe background of an image, with the luminance value corresponding tothe greatest number of pixels being most likely to be a good measure ofthe luminance of the image's background.

Once this background luminance is identified, it can be used to identifythose pixels that correspond to the background, and smooth out theirluminance values so as to reduce the effect of mosquito noise. FIG. 4illustrates step 20, the identification and smoothing of backgroundpixels once the background luminance is known, in further detail. Theprocess of FIG. 4 begins (step 400) with setting a count of pixelnumbers to 1 (step 402), so as to keep track of the pixels processed.The luminance of this first pixel is determined or simply retrieved(step 404), and is compared to the previously-identified backgroundluminance (step 406). If the difference between these two luminancevalues is less than a threshold value, i.e., if the pixel's luminance issufficiently similar to that of the background, then this pixel'sluminance is averaged according to its surrounding pixels. Morespecifically, the surrounding pixels are flagged if they aresufficiently similar to the first pixel (step 408):

$\begin{matrix}{{{flag}\mspace{11mu}\left( {m,n} \right)} = \begin{matrix}{0,} & {{if}\mspace{14mu}{{{Y\;\left( {i,j} \right)} - {Y\;\left( {m,n} \right)}}}} & {\geq \;{FlagThreshold}} \\{1,} & {{if}\mspace{14mu}{{{Y\;\left( {i,j} \right)} - {Y\;\left( {m,n} \right)}}}} & {\leq \;{FlagThreshold}}\end{matrix}} & (3)\end{matrix}$and:

$\begin{matrix}{{{Count}\mspace{11mu}\left( {i,j} \right)} = {\sum\limits_{m = {i - 1}}^{i + 1}{\sum\limits_{n = {j - 1}}^{j + 1}{{flag}\mspace{11mu}\left( {m,n} \right)}}}} & (4)\end{matrix}$where:m=i−1, i, i+1; n=j−1, j, j+1; andi=0, 1, . . . , M−1; j=0, 1, . . . , N−1.

Thus, surrounding pixels are flagged with a “1” if their luminancevalues are within a threshold amount FlagThreshold of the luminance ofpixel (i,j). In some embodiments, a flag threshold value of 10 has beenfound to be effective, although one of ordinary skill in the art willrealize that the invention encompasses the use of any flag thresholdvalue that assists in correctly flagging pixels.

The pixel's luminance is then set equal to the average of the luminancesof that pixel, as well as the flagged pixels (step 410), which has theeffect of smoothing out variances in the luminances of those pixelsbelonging to the background, thus reducing mosquito noise effects:

$\begin{matrix}{{Y^{\prime}\left( {i,j} \right)} = {\frac{\sum\limits_{m = {i - 1}}^{i + 1}{\sum\limits_{n = {j - 1}}^{j + 1}{{flag}\mspace{11mu}\left( {m,n} \right) \times {Y\left( {m,n} \right)}}}}{{Count}\mspace{11mu}\left( {i,j} \right)}\mspace{14mu}{if}}} & (5) \\{{{{Y\left( {i,j} \right)} - {Y({background})}}} \leq {{BackgroundThreshold}\mspace{11mu}.}} & (6)\end{matrix}$On the other hand, if the difference between the luminance value of thispixel and the background luminance is larger than theBackgroundThreshold value of equation (6), the pixel is deemed as notbelonging to the background, and its luminance remains unchanged (step412). In some embodiments, a background threshold value of 10 has beenfound to be effective, although one of skill will realize that theinvention encompasses the use of any energy threshold value that assistsin identifying background luminances.

In either case, the pixel number is then incremented (step 414) and acheck is performed to determine whether any pixels remain (step 416). Ifpixels remain, the process returns to step 304, and continues. If not,the process terminates (step 418).

It should be noted that aspects of the invention include thedetermination of a luminance value corresponding to the background of animage, as well as the identification and averaging/smoothing of pixelsin that background. While the above description illustrates certainmethods for carrying out these aspects, it is not an exhaustive list ofsuch methods. Thus, while the background luminance is identifiedaccording to an “energy” metric identifying pixels that are similar totheir surrounding pixels, the invention need not be limited to thiscase. For instance, the use of different energy threshold values can beemployed. Similarly, different criteria besides energy can be employed,such as visual and/or automatic identification of a background. Also,while background pixels are smoothed according to an average of similarsurrounding pixels, the invention need not be so limited. For example,the invention can utilize various BackgroundThreshold values. It alsoneed not employ a simple numerical average, but can instead smoothpixels according to their neighboring pixels in any known fashion.

The foregoing description, for purposes of explanation, used specificnomenclature to provide a thorough understanding of the invention.However, it will be apparent to one skilled in the art that the specificdetails are not required in order to practice the invention. Thus, theforegoing descriptions of specific embodiments of the present inventionare presented for purposes of illustration and description. They are notintended to be exhaustive or to limit the invention to the precise formsdisclosed. Many modifications and variations are possible in view of theabove teachings. For example, various values of the energy threshold,flag threshold, and background threshold can be employed. Also,background luminances can be identified in a number of ways, includingby an energy metric, other metrics, visually, and/or by any other knownmethod of automatic identification of particular regions of a digitalimage. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical applications,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated.

1. A method of reducing mosquito noise in an image having pixels and arepresentation of a background, comprising: determining a representativeluminance value corresponding to the background, wherein therepresentative luminance value is a function of a comparison between aluminance value of a selected pixel in the image with a set of adjacentluminance values corresponding to luminance values of pixels that areproximate to the selected pixel; comparing luminance values of thepixels to the representative luminance value corresponding to thebackground, so as to determine a set of background pixels; and assigningto pixels of the set of background pixels average luminance valuesdetermined according to luminance values of adjacent pixels.
 2. Themethod of claim 1 wherein determining a representative luminance valuecorresponding to the background further comprises: (A) receiving theluminance value of the selected pixel; (B) determining the adjacentluminance values corresponding to luminance values of pixels that areproximate to the selected pixel; (C) calculating a difference valueaccording to differences between the luminance value of the selectedpixel and the adjacent luminance values; (D) comparing the differencevalue to a difference value threshold; (E) if the difference valueexceeds the difference value threshold, incrementing an element of anarray of luminance values, the element corresponding to the luminancevalue of the selected pixel; and (F) selecting the luminance valuecorresponding to the numerically largest element of the elements, so asto select the luminance value corresponding to the background.
 3. Themethod of claim 2 further comprising, prior to (F), repeating (A), (B),(C), (D), and (L) for each of the pixels.
 4. The method of claim 2wherein (C) further comprises setting the difference value equal to anaverage of the differences between the luminance value of the selectedpixel and the adjacent luminance values.
 5. The method of claim 2wherein the difference value threshold is about
 15. 6. The method ofclaim 5 wherein the assigning further comprises leaving the luminancevalue of the selected pixel unchanged when the difference thresholdvalue does not exceed the luminance difference corresponding to thedifference between the luminance value of the selected pixel and therepresentative luminance value corresponding to the background.
 7. Themethod of claim 1: wherein the comparing further comprises computingluminance differences between the luminance values of the pixels and therepresentative luminance value corresponding to the background, andcomparing the luminance differences to a first luminance differencethreshold; and wherein the assigning further comprises assigning theaverage luminance values when the first luminance difference thresholdexceeds the luminance differences.
 8. The method of claim 7 wherein thefirst luminance difference threshold is about
 10. 9. The method of claim7 wherein the assigning further comprises: (A) determining the luminancevalues of the selected pixel and of the pixels adjacent to the selectedpixel; (B) flagging those pixels of the pixels adjacent to the selectedpixel that have luminance values that are different from the luminancevalue of the selected pixel by an amount less than a second luminancedifference threshold; and (C) setting the luminance value of theselected pixel substantially equal to the average of the luminancevalues of the flagged pixels.
 10. The method of claim 9 wherein thesecond luminance difference threshold is about
 10. 11. A computerreadable medium encoded with computer-executable instructions forperforming a method for reducing mosquito noise in an image havingpixels and a representation of a background, the method comprising: afirst set of instructions for determining a representative luminancevalue corresponding to the background, wherein the representativeluminance value is a function of a comparison between a luminance valueof a selected pixel in the image with a set of adjacent luminance valuescorresponding to luminance values of pixels that are proximate to theselected pixel; a second set of instructions for comparingrepresentative luminance values of the pixels to the luminance valuecorresponding to the background, so as to determine a set of backgroundpixels; and a third set of instructions for assigning to pixels of theset of background pixels average luminance values determined accordingto luminance values of adjacent pixels.
 12. The computer readable mediumof claim 11 wherein the first set of instructions further comprisesinstructions for: (A) determining the luminance value of the selectedpixel; (B) determining the adjacent luminance values corresponding toluminance values of pixels that are proximate to the selected pixel; (C)calculating a difference value according to differences between theluminance value of the selected pixel and the adjacent luminance values;(D) comparing the difference value to a difference value threshold; (E)if the difference value exceeds the difference value threshold,incrementing an element of an array of luminance values, the elementcorresponding to the luminance value of the selected pixel; and (F)selecting the luminance value corresponding to the numerically largestelement of the elements, so as to select the luminance valuecorresponding to the background.
 13. The computer readable medium ofclaim 12 wherein the first set of instructions further comprises, priorto (F), repeating (A), (B), (C), (D), and (E) for each of the pixels.14. The computer readable medium of claim 12 wherein (C) furthercomprises instructions for setting the difference value equal to anaverage of the differences between the luminance value of the selectedpixel and the adjacent luminance values.
 15. The computer readablemedium of claim 12 wherein the difference value threshold is about 15.16. The computer readable medium of claim 15 wherein the third set ofinstructions further comprises instructions for leaving the luminancevalue of the selected pixel unchanged when the difference thresholdvalue does not exceed the luminance difference corresponding to thedifference between the luminance value of the selected pixel and therepresentative luminance value corresponding to the background.
 17. Thecomputer readable medium of claim 11: wherein the second set ofinstructions further comprises instructions for computing luminancedifferences between the luminance values of the pixels and therepresentative luminance value corresponding to the background, andcomparing the luminance differences to a first luminance differencethreshold; and wherein the third set of instructions further comprisesinstructions for assigning the average luminance values when the firstluminance difference threshold exceeds the luminance differences. 18.The computer readable medium of claim 17 wherein the first luminancedifference threshold is about
 10. 19. The computer readable medium ofclaim 17 wherein the third set of instructions further comprisesinstructions for: (A) determining the luminance values of the selectedpixel and of the pixels adjacent to the selected pixel; (B) flaggingthose pixels of the pixels adjacent to the selected pixel that haveluminance values that are different from the luminance value of theselected pixel by an amount less than a second luminance differencethreshold; and (C) setting the luminance value of the selected pixelsubstantially equal to the average of the luminance values of theflagged pixels.
 20. The computer readable medium of claim 19 wherein thesecond luminance difference threshold is about 10.